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Abstract 



We prove that every graph has a spectral sparsifier with a number of edges lin- 
ear in its number of vertices. As linear-sized spectral sparsifiers of complete graphs 
are expanders, our sparsifiers of arbitrary graphs can be viewed as generalizations of 
expander graphs. 

In particular, we prove that for every d > 1 and every undirected, weighted graph 
G = (V, E, w) on n vertices, there exists a weighted graph H = (V, F, w) with at most 
\d(n — l)] edges such that for every x £ R , 



where Lq and Lh are the Laplacian matrices of G and H, respectively. Thus, H 
approximates G spectrally at least as well as a Ramanujan expander with dn/2 edges 
approximates the complete graph. 

We give an elementary deterministic polynomial time algorithm for constructing 



1 Introduction 

A sparsifier of a graph G = (V, E, w) is a sparse graph H that is similar to G in some useful 
way. Many notions of similarity have been considered. For example, Chew's [6] spanners 
have the property that the distance between every pair of vertices in H is approximately the 
same as in G. Benczur and Karger's [3] cut-sparsifiers have the property that the weight of 
the boundary of every set of vertices is approximately the same in G as in H. We consider 
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the spectral notion of similarity introduced by Spielman and Teng [T7J [12] : we say that H 
is a k- approximation of G if for all x G M. v , 

x T LqX < x T Lhx < k ■ x 1 LqX, (1) 

where Lq and Lh are the Laplacian matrices of G and H . We recall that 

X LqX ^ VJu,v{Xu Xv) i 

(u,v)eE 

where w U)V is the weight of edge (u, v) in G. By considering vectors x that are the character- 
istic vectors of sets, one can see that condition ([!]) is strictly stronger than the cut condition 
of Benczur and Karger. 

In the case where G is the complete graph, excellent spectral sparsifiers are supplied by 
Ramanujan Graphs [T5] [T5] . These are <i-regular graphs H all of whose non-zero Laplacian 
eigenvalues lie between d — 2y ' d — 1 and d + 2y ' d — 1. Thus, if we take a Ramanujan graph 
on n vertices and multiply the weight of every edge by n/(d — 2yd — 1), we obtain a graph 
that ^-approximates the complete graph, for 



d + 2y/d~ 

K 



d-2Vd^l' 



In this paper, we prove that every graph can be approximated at least this welQ by a 
graph with only twice as many edges as the Ramanujan graph (as a d-regular graph has 
dn/2 edges). 

Theorem 1.1. For every d > 1, every undirected weighted graph G = (V, E, w) on n vertices 
contains a weighted subgraph H = (V,F,w) with \d(n — 1)] edges (i.e., average degree at 
most 2d) that satisfies: 

x T L G x < x T L H x < ( d + 1 + 2 ^ I . x T L G x Vx G R v . 
\d+l-2VdJ 

Our proof provides a deterministic greedy algorithm for computing the graph H in time 
0(dn 3 m). 

We remark that while the edges of H are a subset of the edges of G, the weights of edges 
in H and G will typically be different. In fact, there exist unweighted graphs G for which 
every good spectral sparsifier H must contain edges of widely varying weights [19J. 



1 Strictly speaking, our approximation constant is only better than the Ramanujan bound k 



in the regime d > ■ This includes the actual Ramanujan graphs, for which d is an integer greater than 
2. 
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1.1 Expanders: Sparsifiers of the Complete Graph 



In the case that G is a complete graph, our construction produces expanders. However, these 
expanders are slightly unusual in that their edges have weights, they may be irregular, and 
the weighted degrees of vertices can vary slightly This may lead one to ask whether they 
should really be considered expanders. In Section [4] we argue that they should be. 

As the graphs we produce are irregular and weighted, it is also not immediately clear 
that we should be comparing k with the Ramanujan bound of 

It is knownj^jthat no <i-regular graph of uniform weight can ^-approximate a complete graph 
for k asymptotically better than ^ |14j . While we believe that no graph of average degree 
d can be a k- approximation of a complete graph for k asymptotically better than ([2]), we are 
unable to show this at the moment and prove instead the weaker claim that no such graph 
can achieve n less than 

Vd \ n 



1.2 Prior Work 

Spielman and Teng [T7J US] introduced the notion of sparsification that we consider, and 
proved that (1 + e)- approximations with O (n/e 2 ) edges could be constructed in O (m) time. 
They used these sparsifiers to obtain a nearly-linear time algorithm for solving diagonally 
dominant systems of linear equations [I7J HE] . 

Spielman and Teng were inspired by the notion of sparsification introduced by Benczur 
and Karger [3] for cut problems, which only required inequality (flj to hold for all x E 
{0, 1} V ■ Benczur and Karger showed how to construct graphs H meeting this guarantee 
with O (nlogn/e 2 ) edges in O (mlog 3 n) time; their cut sparsifiers have been used to obtain 
faster algorithms for cut problems [31 |TT] . 

Spielman and Srivastava [16] proved the existence of spectral sparsifiers with O {n log n/e 2 ) 
edges, and showed how to construct them in O (m) time. They conjectured that it should 
be possible to find such sparsifiers with only 0(n/e 2 ) edges. We affirmatively resolve this 
conjecture. 

Recently, partial progress was made towards this conjecture by Goyal, Rademacher and 
Vempala [9], who showed how to find graphs H with only 2n edges that O (log n)-approximate 
bounded degree graphs G under the cut notion of Benczur and Karger. 

We remark that all of these constructions were randomized. Ours is the first deterministic 
algorithm to achieve the guarantees of any of these papers. 

2 While lower bounds on the spectral gap of c£-regular graphs focus on showing that the second-smallest 
eigenvalue is asymptotically at most d — 2\J d — 1, the same proofs by test functions can be used to show 
that the largest eigenvalue is at asymptotically least d + 2^/d — 1. 
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2 Preliminaries 



2.1 The Incidence Matrix and the Laplacian 

Let G = (V, E, w) be a connected weighted undirected graph with n vertices and m edges 
and edge weights w e > 0. If we orient the edges of G arbitrarily, we can write its Laplacian 
as L = B T WB, where B mxn is the signed edge-vertex incidence matrix, given by 

{1 if v is e's head 
— 1 if v is e's tail 
otherwise 

and W mxm is the diagonal matrix with W(e,e) = w e . It is immediate that L is positive 
semidefmite since: 

x T Lx = x T B T WBx = \\W 1/2 Bx\\l 

= w u ,v{x u — x v) 2 > 0, for every x G lR n . 

(u,v)eE 

and that G is connected if and only if ker(L) = ker(W 1 ^ 2 B) = span(l). 

2.2 The Pseudoinverse 

Since L is symmetric we can diagonalize it and write 

n-1 

1=1 

where Ai, . . . , X n ~i are the nonzero eigenvalues of L and u\, . . . , u n _\ are a corresponding set 
of orthonormal eigenvectors. The Moore-Penrose Pseudoinverse of L is then defined as 

n—l j 

i=l Xi 

Notice that ker(L) = ker(L + ) and that 

n-1 

LL+ = L + L = J2 u i u f' 

i=i 

which is simply the projection onto the span of the nonzero eigenvectors of L (which are also 
the eigenvectors of L + ). Thus, LL + = L + L is the identity on im(L) = ker(L)- 1 . 
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2.3 Formulas for Rank-one Updates 

We use the following well-known theorem from linear algebra, which describes the behavior 
of the inverse of a matrix under rank-one updates (see [H Section 2.1.3]). 

Lemma 2.1 (Sherman-Morrison Formula). If A is a nonsingular n x n matrix and v is a 
vector, then 

(A + vv T - 1 = A - - - - . 

There is a related formula describing the change in the determinant of a matrix under 
the same update: 

Lemma 2.2 (Matrix Determinant Lemma). If A is nonsingular and v is a vector, then 

det{A + vv T ) = det(A)(l + v T A~ 1 v). 

3 The Main Result 

At the heart of this work is the following purely linear algebraic theorem. We use the notation 
A ^ B to mean that B — A is positive semidefmite, and ids to denote the identity operator 
on a vector space S. 

Theorem 3.1. Suppose d > 1 and vi, V2, • • • , v m are vectors in W 1 with 

i<m 

Then there exist scalars Sj > u>z£/i \{i : Sj 7^ 0}| < dn so that 



x ^ T fd+l + 2\Q\ . , 

> , SiVjV^ ^ = id n 

^ \d + l-2y/d) 

i<m \ / 



The sparsification result for graphs follows quickly from this theorem as shown below. 

Proof of Theorem Assume without loss of generality that G is connected. Write Lq = 
B T WB as in Section [2.l| and fix d > 1. Restrict attention to im(La) — M n_1 and apply 



Theorem 3.1 to the columns {vj}j< m of 

V nxm = (L+)^B T W 1 i, 
which are indexed by the edges of G and satisfy 

J^VivJ = VV T = (L^B T WB(L+^ 

= (L+)iL G (L+)5 = id im(LG) . 
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Write the scalars Sj > guaranteed by the theorem in the mxm diagonal matrix S(i, i) = Sj 
and set Lh = B T W^ SW^ B. Then Lh is the Laplacian of the subgraph H of G with edge 
weights {uii = WiSi}i £ E, and H has at most d(n — 1) edges since at most that many of the 
Si are nonzero. Also, 



id im (L G ) d> s i v i v f = VSV T di k ■ id im (L G ) for k = - + 1 + 

~r a + 1 — 2yd 

By the Courant-Fischer Theorem, this is equivalent to: 

v T VSV T v i 
1 < y < k Wye im((L G )5) = im(L G ) 



y T {L+)lL H {L%)h 

y T y 



1 < T ^ — 5=1 < K Vy G im((L G 



1 < GV g; , ; g; — ^- < k Va; JL 1 

ryi J. f 2 T 2 

X IjqIjqX 

1 < < K vx ± 1 



x T L G a; 

as desired. □ 

It is worth mentioning that the above reduction is essentially the same as the one in 
[15] . In that paper, the authors consider the symmetric projection matrix II = BLqB t 
whose columns {IT e } ee £ correspond to the edges of G. They show, by a concentration 
lemma of Rudelson [15], that randomly sampling O(nlogn) of the columns with probabilities 
proportional to ||II e || 2 = u> e R eff (e) (where R eff is the effective resistance) gives a matrix IT 
that approximates II in the spectral norm and corresponds to a graph sparsifier, with high 
probability. In this paper, we do essentially the same thing with two modifications: we 
eliminate II in order to simplify notation, since we are no longer following the intuition 
of sampling by effective resistances; and, instead of Rudelson's sampling lemma, we use 



Theorem 3.1 to deterministically select 0(n) edges (equivalently, columns of II). 

The rest of this section is devoted to proving Theorem |3.1| The proof is constructive and 
yields a deterministic polynomial time algorithm for finding the scalars Sj, which can then 
be used to sparsify graphs, as advertised. 

Given vectors {vj}, our goal is to choose a small set of coefficients Sj so that A = 
s{VivJ is well-conditioned. We will build the matrix A in steps, starting with A = and 
adding one vector SjVjV^ 1 at a time. Before beginning the proof, it will be instructive to study 
how the eigenvalues and characteristic polynomial of a matrix evolve upon the addition of 
a vector. This discussion should provide some intuition for the structure of the proof, and 
demystify the origin of the ' Twice- Ramanuj an' number j^^vll wn i cn appears in our final 
result. 
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3.1 Intuition for the Proof 



It is well known that the eigenvalues of A + vv T interlace those of A. In fact, the new 
eigenvalues can be determined exactly by looking at the characteristic polynomial of v4+vv T , 



which is computed using Lemma 2.2 as follows 



Pa+w t (%) — det(xl — A — vv ] 



Pa{x) 



x - Xi 



where A, are the eigenvalues of A and Uj are the corresponding eigenvectors. The polynomial 
Pa +vv t(x) has two kinds of zeros A: 

1. Those for which Pa{X) = 0. These are equal to the eigenvalues Xj of A for which the 
added vector v is orthogonal to the corresponding eigenvector Uj, and which do not 
therefore 'move' upon adding vv T . 

2. Those for which p^(A) 7^ and 



/(A) 



A-A, 



0. 



These are the eigenvalues which have moved and strictly interlace the old eigenvalues. 
The above equation immediately suggests a simple physical model which gives intuition 
as to where these new eigenvalues are located. 



(v,u„) 2 = l/4 




Figure 1: Physical model of interlacing eigenvalues. 



Physical Model. We interpret the eigenvalues A as charged particles lying on a slope. 
On the slope are n fixed, chargeless barriers located at the initial eigenvalues Xj, and 
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each particle is resting against one of the barriers under the influence of gravity. Adding 
the vector vv T corresponds to placing a charge of (v, Uj) 2 on the barrier corresponding 
to Xj. The charges on the barriers repel those on the eigenvalues with a force that is 
proportional to the charge on the barrier and inversely proportional to the distance 
from the barrier — i.e., the force from barrier j is given by 

(v»%) 2 
A-V 

a quantity which is positive for Xj 'below' A, which are pushing the partical 'upward', 
and negative otherwise. The eigenvalues move up the slope until they reach an equi- 
librium in which the repulsive forces from the barriers cancel the effect of gravity, 
which we take to be a +1 in the downward direction. Thus the equilibrium condition 
corresponds exactly to having the total 'downward pull' /(A) equal to zero. 

With this physical model in mind, we begin to consider what happens to the eigenvalues 
of A when we add a random vector from our set {vj}. The first observation is that for any 
eigenvector Uj (in fact for any vector at all), the expected projection of a randomly chosen 
v E {vi}i< m is 

E v (v,w i ) 2 = -V(v ! ,tt 3 } 2 = -uj I Vvivfj Uj = It^lL = — . 

m / — ' m J y~ J 771 m 

Of course, this does not mean that there is any single vector in our set that realizes this 
'expected behavior' of equal projections on the eigenvectors. But if we were to add such a 
vector^] in our physical model, we would add equal charges of 1/m to each of the barriers, 
and we would expect all of the eigenvalues of A to drift forward 'steadily'. In fact, one might 
expect that after sufficiently many iterations of this process, the eigenvalues would all march 
forward together, with no eigenvalue too far ahead or too far behind, and we would end up 
in a position where X max / X m i n is bounded. 

In fact, this intuition turns out to be correct. Adding a vector with equal projections 
changes the characteristic polynomial in the following manner: 




= Pa(x) - (l/m)p' A (x), 



since p'^ix) = J2j Yli^j( x ~ ^«)- ^ we start with A = 0, which has characteristic polynomial 
Po(x) = x n , then after k iterations of this process we obtain the polynomial 

p k (x) = (I - (l/m)D) k x n 

3 For concreteness, we remark that this 'average' vector would be precisely 

V avg = -Uy^j- 

j 
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where D is the derivative with respect to x. Fortunately, iterating the operator (/ — aD) for 
any a > generates a standard family of orthogonal polynomials - the associated Laguerre 
polynomials [?j . These polynomials are very well-studied and the locations of their zeros are 
known; in particular, after k = dn iterations the ratio of the largest to the smallest zero is 
known [7] to be 

d+l + 2y/d 
d + 1 -2Vd' 

which is exactly what we want. 

To prove the theorem, we will show that we can choose a sequence of actual vectors that 
realizes the expected behavior (i.e. the behavior of repeatedly adding v avg ), as long as we 
are allowed to add arbitrary fractional amounts of the v{vf via the weights Si > 0. We will 
control the eigenvalues of our matrix by maintaining two barriers as in the physical model, 
and keeping the eigenvalues between them. The lower barrier will 'repel' the eigenvalues 
forward; the upper one will make sure they do not go too far. The barriers will move 
forward at a steady pace. By maintaining that the total 'repulsion' at every step of this 
process is bounded, we will be able to guarantee that there is always some multiple of a 
vector to add that allows us to continue the process. 

3.2 Proof by Barrier Functions 

We begin by defining two 'barrier' potential functions which measure the quality of the 
eigenvalues of a matrix. These potential functions are inspired by the inverse law of repulsion 
in the physical model discussed in the last section. 

Definition 3.2. For u,l G E and A a symmetric matrix with eigenvalues Ai, A 2 , . . . , \ n , 

define: 

$ U (A) = Tt(uI - A)' 1 = ~TT ( U PP er Potential). 

i 

$i(A) = Tr(A - II)- 1 = V (Lower potential). 

\ — I 

As long as A -< ul and A y II (i.e., \ max (A) < u and \ ni \ n (A) > I), these potential 
functions measure how far the eigenvalues of A are from the barriers u and /. In particular, 
they blow up as any eigenvalue approaches a barrier, since then uI — A (or A — II) approaches 
a singular matrix. Their strength lies in that they reflect the locations of all the eigenvalues 
simultaneously: for instance, Q U (A) < 1 implies that no Aj is within distance one of u, no 2 
Aj's are at distance 2, no k are at distance k, and so on. In terms of the physical model, the 
upper potential $ U (A) is equal to the total repulsion of the eigenvalues of A from the upper 
barrier u, while <&i(A) is the analogous quantity for the lower barrier. 

To prove the theorem, we will build the sum SjVjV^ iteratively, adding one vector at 
a time. Specifically, we will construct a sequence of matrices 

o = a(°\aV,...,aw 
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along with positive constantqjw , lo, o~u, £u and e L which satisfy the following conditions: 

(a) Initially, the barriers are at u = uq and I = /o and the potentials are 

^(A^) = eu and ^ lo {A^) = e L . 

(b) Each matrix is obtained by a rank-one update of the previous one — specifically by 
adding a positive multiple of an outer product of some Vj. 

A (g+i) = A (q) + tvv T for gome v G {vj} and t > 0. 

(c) If we increment the barriers u and I by Su and 5l respectively at each step, then the 
upper and lower potentials do not increase. For every q = 0, 1, ... Q, 

(4(9+1)) < < e[/ for u = UQ + q s v . 

$ l+ s L (A^) < < e L for / = / + g5 L . 

(d) No eigenvalue ever jumps across a barrier. For every q = 0, 1, . . . Q, 

A ma x(^ (g) ) <u + qdu and A min (A {g) ) > l Q + g<5 L . 

To complete the proof we will choose uq, lo, 5jj, 5l, £jj and so that after Q = dn steps, the 
condition number of is bounded by 

A max (^ (Q) ) mq + rfn^ _ rf + 1 + 2y/rf 

A min 

(AW)) - l + dn5 L d+l-2Vd' 

By construction, is a weighted sum of at most dn of the vectors, as desired. 

The main technical challenge is to show that conditions (b) and (c) can be satisfied 
simultaneously — i.e., that there is always a choice of vv T to add to the current matrix 
which allows us to shift both barriers up by a constant without increasing either potential. 
We achieve this in the following three lemmas. 

The first lemma concerns shifting the upper barrier. If we shift u forward to u+Sjj without 
changing the matrix A, then the upper potential $ U (A) decreases since the eigenvalues Aj 
do not move and u moves away from them. This gives us room to add some multiple of a 
vector tvv T , which will move the \ towards I and increase the potential, counteracting the 
initial decrease due to shifting. The following lemma quantifies exactly how much of a given 
vv T we can add without increasing the potential beyond its original value before shifting. 



On first reading the paper, we suggest the reader follow the proof with the assignment ejj = = 1, 
uq = n, Iq = —n, 8jj = 2, 5l — 1/3. This will provide the bound (6d+ l)/(d— 1), and eliminates the need 



to use Claim 3.6 
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Lemma 3.3 (Upper Barrier Shift). Suppose A max (A) < u, and v is any vector. If 

1 v T ((w + 5tj)I — AY 2 w T .. . . r A . , dcf TT , x 
7 > 1 / ,x i ti Ax + v T ((u + S V )I - A) v = U A (v) 

then 

$ u+Su (A + tvv T ) < and A max (A + tvv T ) < u + 5 V . 

That is, if we add t times vv T to A and shift the upper barrier by 5jj, then we do not increase 
the upper potential. 

We remark that Ua(v) is linear in the outer product vv T . 

Proof. Let u' — u + Su- By the Sherman-Morrison formula, we can write the updated 
potential as: 

- Tr (in' I - AY 1 + t(u'I - A)-^(u'I - A)^ \ 

-lr(ui-A) + i _ tvT(M// _ A)lv 

since Tr is linear and Tr(XF) = Tr(FX) 
, ,n tv T (w7 - A)~ 2 v 

K J 1 - tV T ( M '/ - A)-! V 

. w - (*-(>i) - y^oD) + 1/ ;_ T( ^;/_ ) ";;., v 



As Ua(v) > v T {u'I — A) 1 v, the last term is finite for 1/t > U A iy)- By now substituting 
any 1/t > U A {v) we find $ M+<5c/ (A + tvv T ) < This also tells us that A max (A + tvv T ) < 

u + 5(7, as if this were not the case, then there would be some positive t' < t for which 
A ma x(^4 + t'vv T ) =u + 5jj. But, at such a t', & u+5u (A + t'vv T ) would blow up, and we have 
just established that it is finite. □ 

The second lemma is about shifting the lower barrier. Here, shifting / forward to I + Sl 
while keeping A fixed has the opposite effect — it increases the lower potential &i(A) since 
the barrier / moves towards the eigenvalues Aj. Adding a multiple of a vector tvv T will 
move the Aj forward and away from the barrier, decreasing the potential. Here, we quantify 
exactly how much of a given vv T we need to add to compensate for the initial increase from 
shifting /, and return the potential to its original value before the shift. 

Lemma 3.4 (Lower Barrier Shift). Suppose \ min (A) > I, $i(A) < 1/Sl, and v is any vector. 
If 



11 



then 



$i+5 L ( A + ivvT ) < M A ) and \ mill (A + tvv T ) > I + S L . 



That is, if we add t times vv T to A and shift the lower barrier by 5l, then we do not increase 
the lower potential. 

Proof. First, observe that A min (A) > I and $i(A) < 1/Sl imply that X min (A) > I + Sl- So, 
for every t > 0, A m ; n (A + tvv T ) > I + 5l- 

Now proceed as in the proof for the upper potential. Let I' = I + 5l- By Sherman- 
Morrison, we have: 



The third lemma identifies the conditions under which we can find a single tvv T which 
allows us to maintain both potentials while shifting barriers, and thereby continue the pro- 
cess. The proof that such a vector exists is by an averaging argument, so this can be seen 
as the step in which we relate the behavior of actual vectors to the behavior of the expected 
vector v avg . Notice that the use of variable weights t, from which the eventual Si arise, is 
crucial to this part of the proof. 

Lemma 3.5 (Both Barriers). If X max (A) < u, A min (A) > I, $ M (A) < e U} $i(A) < e L , and 
eu,eL,8u and 5l satisfy 



$i + 8 L (A + tvv T ) = Tr(A + tvv T - Vl)' 1 




l + 5 ^ A > l + tv T {A-l'I)-H 




Rearranging shows that &i + s L (A + tvv T ) < $i(A) when 1/t < L A (v). 



□ 



< — + e v < — - t L 
On o L 



(3) 



then there exists an i and positive t for which 

L A (yi) > 1/t > U A (vi), X max (A + tVivf) <u + 5u, 
and \ m - m (A + tvjvf ) > 1 + 5 L . 



Proof. We will show that 



EMvi)>EMvi), 
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from which the claim will follow by Lemmas 3.3 and |3.4| We begin by bounding 



£ ^<v.) = El ^^ + £ vf ((„ + *,), - A)- V, 



+ Tr((u + 5^/)/ - A)" 1 



Tr( (it + 5y)I - A)~ 
since Vjvf = / and X • I = Tr(X) 

i 



E^-a^-E^ + ^-a 



L 

SuYnfa-^-Hu + Su-Xi)- 1 1 J 

as J2( u - + $u - ^y 1 > J2( u + 6u ~ A *)~ 2 

i i 

< 1/5 V + $ U {A) < l/6 v + e v . 
On the other hand, we have 



^-(z + wrME^vf 



-(A-(/ + 5 L )/)- 1 . ^v.vf 



Tr(A-(/ + ^ L )/)- 2 



since 



^ Vi vf = / and X • I = Tr(X) 



Ei(Ai - I - h)~ x - Ei(A» - 
> 1/<5 L - ^(A, - Z)- 1 = 1/<5 L - e L 



by Claim 3.6 



Putting these together, we find that 



as desired. 



□ 
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Claim 3.6. If \>l for all i, < £\(A; - l)' 1 < e L , and 1/5 L - e L > 0, i/jen 

- j - ^)- 2 v i 

EiCA* - z - h)- 1 - EA - _1 ^k-i-h 

Proof. We have 

5l < l/e L < A< - Z, 

for every z. So, the denominator of the left-most term on the left-hand side is positive, and 
the claimed inequality is equivalent to 



i>-<-^ (e^-e^) (^ + e^-e^ 

* l ?(a,-z-L)(a,-o) + 

^ ( Xl - I - 5 L )(\ - I) + ( 6l ^ 



A,-Z 

i 

1 

(Xi-l - S L )(Xi -I) 

2 



(Ai — Z — - /) 

which, by moving the first term on the RHS to the LHS, is just 



(Xi-l-S L ) 2 (\i-l) - \ ^ {Xi-l - 5 L )(Xi - I) 
By Cauchy-Schwartz, 



E(a,-o _1 <^ 



(A 4 -/-5 L ) 2 (A 4 -Z) 
l) 2 (A,-/) 



< 



since 

1 



(Ai-Z-^Ai-Z) 



1 

since > 0, 



and so (El) is established. □ 
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Proof of Theorem 3.1. All we need to do now is set eu,eL,8u, and Sl in a manner that 
satisfies Lemma 3J3 and gives a good bound on the condition number. Then, we can take 
= and construct A^ 1 ^ from A^ by choosing any vector v, with 

L A ( q) (-Vi) > U A („)(vi) 



(such a vector is guaranteed to exist by Lemma 3.5) and setting A^ 1 ^ = A^ + t\{vj for 
any t > satisfying: 

L A ( q )(vi) > - > U A ( q )(vi). 

It is sufficient to take 

6l = 1 eL = ~^2 l = -n/e L 

y/d + l y/d-l . 

ou = -t= — - e n = 7= = n/exj. 

yd — 1 a + v« 

We can check that: 

1 - 1 - 1 ! 1 1 

^ 6(7 ~ + 1 v / rf(v / rf+ 1) ~ S~l e ^ 

so that (|3l is satisfied. 

* — ' n 

The initial potentials are $ e c (0) = €[/ and $js_(0) = e^. After dn steps, we have 

A m ax(^")) < n/e V + dTKJt; 



A mi „(AW) - -n/e L + dn5 L 

d — v^d 
_ d + 2 v / d + 1 
d - 2^ + 1 ' 

as desired. □ 

To turn this proof into an algorithm, one must first compute the vectors Vj, which can be 
done in time O (n 2 m). For each iteration of the algorithm, we must compute ((u+Su)I— , 
((u + 5jj) I — A)~ 2 , and the same matrices for the lower potential function. This computation 
can be performed in time O (n 3 ). Finally, we can decide which edge to add in each iteration 
by computing U A (vi) and L A (yi) for each edge, which can be done in time O (n 2 m). As we 
run for dn iterations, the total time of the algorithm is O (dn 3 m). 
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4 Sparsifiers of the Complete Graph 



Let G = (V, E) be the complete graph on n vertices, and let H = (V, F, w) be a weighted 
graph of average degree d that (1 + e)-approximates G. As x T L G x = n\\x\\ 2 for every x 
orthogonal to 1, it is immediate that every vertex of H has weighed degree between n and 
(1 + e)n. Thus, one should think of H as being an expander graph in which each edge weight 
has been multiplied by n/d. 

As H is weighted and can be irregular, it may at first seem strange to view it as an 
expander. However, it may easily be shown to have the properties that define expanders: 
it has high edge-conductance, random walks mix rapidly on H and converge to an almost- 
uniform distribution, and it satisfies the Expander Mixing Property (see [2] or [TUl Lemma 
2.5]). High edge-conductance and rapid mixing would not be so interesting if the weighted 
degrees were not nearly uniform — for example, the star graph has both of these properties, 
but the random walk on the star graph converges to a very non-uniform distribution, and 
the star does not satisfy the Expander Mixing Property. For the convenience of the reader, 
we include a proof that H has the Expander Mixing Property below. 

Lemma 4.1. Let Lh = (V,E,w) be a graph that (1 + e)- approximates L G , the complete 
graph on V. Then, for every pair of disjoint sets S and T, 



w(S,T) - (l + \S\ \T\\ < n(e/2)^\S\[Tl 



2, 

where w(S,T) denotes the sum of the weights of edges between S and T. 
Proof. We have 

so we can write 

L H =(l + |) L G + M, 

where M is a matrix of norm at most (e/2) \\Lg\\ < ne/2. Let x be the characteristic vector 
of S, and let y be the characteristic vector of T. We have 

-w(S,T) = x T L H y. 

As G is the complete graph and S and T are disjoint, we also know 

x T L G y = -\S\ \T\. 

Thus, 



x 1 L H y = ( 1 + - ) x 1 L G y + x 1 My 



= -(l + |) \S\\T\+x T My. 

The lemma now follows by observing that 

x T My < \\M\\ \\x\\ \\y\\ < n(e/2)y/\S\ \T\. 



□ 
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Using the proof of the lower bound on the spectral gap of Alon and Boppana (see |14j ) 
one can show that a <i-regular unweighted graph cannot ^-approximate a complete graph 
for k asymptotically better than ([2]). We conjecture that this bound also holds for weighted 
graphs of average degree d. Presently, we prove the following weaker result for such graphs. 

Proposition 4.2. Let G be the complete graph on vertex set V , and let H = (V,E,w) be a 
weighted graph with n vertices and a vertex of degree d. If H k- approximates G, then 



Proof. We use a standard approach. Suppose if is a ^-approximation of the complete graph. 
We will construct vectors x* and y* orthogonal to the 1 vector so that 

y* T L H y* \\x*\\ 2 
x* t Lhx* ||y*|| 2 

is large, and this will give us a lower bound on k. 

Let vo be the vertex of degree d, and let its neighbors be V\,...,Vd- Suppose Vi is 
connected to Vq by an edge of weight Wi, and the total weight of the edges between V{ and 
vertices other than Vq, v i, . . . , v d is <5j. We begin by considering vectors x and y with 



These vectors are not orthogonal to 1, but we will take care of that later. It is easy to 
compute the values taken by the quadratic form at x and y: 





for u = v , 

for u = Vi, i > 1, 

for u £ {v , ...,v d } 




for u = t> , 

for u — Vi, i > 1, 

for u {v , ...,v d } 



d d 




mil - 1/v 7 ^) 2 + - o) 2 



i=i i=i 



d 



d 




and 



d d 



y T L H y 



WiO- + v^) 2 + Yl w-i/^d - o) 2 




d d d 



/Vd. 
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The ratio in question is thus 

V T L H y _ Ei w i + Ei(^ + Wi)/d + 2 ]T . Wjj y/d 

T L H x ~J2 iWi + Zitfi + w t )/d - 2 £\ Wi /Vd 



x 



i I J 2 E^» 

s/dT, i ^T, i i.s i +m)/d 



1 



2E, »gj 



Si U'i+SiC^+^i)/ 6 ' 

Since if is a ^-approximation, all weighted degrees must lie between n and nK, which gives 

2E*«>i 2 2 



Therefore, 



Ei^i + + m)/d i + Efe^j/j - i + «■ 

Ei w » 

Let x* and be the projections of x and y respectively orthogonal to the 1 vector. Then 

II *l|2 || 112 I M r\2 o (! + V 7 ^) 2 

II = ll x ll —\ x ,*-/v n ) =2 



n 



and 



so that as n — ► oo 



I *l|2 || 112 / n / o ( X ~ 

\y II = IMI - (!/.Vvn) = 2 - 



n 



113/ II V n 

Combining (|5| and ([6]), we conclude that asymptotically: 



|z*|| 2 



x* T L H x* \\y*\\ 2 ~ 1 - ^j^. \ \ n 
But by our assumption the LHS is at most k, so we have 

1 Vdi+K- V V 



which on rearranging gives 



2 ^ ( Vd 

K>l + -=-0[ — 



as desired. □ 
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5 Conclusion 



We conclude by drawing a connection between Theorem 3^ and an outstanding open problem 
in mathematics, the Kadison-Singer conjecture. This conjecture, which dates back to 1959, 
is equivalent to the well-known Paving Conjecture [TJ [3] as well as to a stronger form of the 
restricted invertibility theorem of Bourgain and Tzafriri jH |S] . The following formulation is 
due to Nik Weaver EDI. 



Conjecture 5.1. There are universal constants e > 0,5 > 0, and r e N for which the 
following statement holds. If v 1; . . . , v m e M. n satisfy ||vj|| < 5 for all i and 



E^. r 



then there is a partition X\, . . . X r of {1, ... , m} for which 



E v.v, 5 



ieXj 



< 1 



for every j 



Suppose we had a version of Theorem 3.1 which, assuming Vj < 6, guaranteed that 



the scalars s» were all either or some constant /3 > 0, and gave a constant approximation 
factor k < /3. Then we would have 



ie5 

for S 1 = {« : Si 7^ 0}, yielding a proof of Conjecture 
since 

K 



5.1 



with r = 2 and e = min{l — ^, 4} 



5>W 



< — < 1 — e and 



ie5 



A, 



< 1 



J65 



/3 



< 1 



e. 



As a special case, such a theorem would also imply the existence of unweighted sparsifiers 
for the complete graph and other (sufficiently dense) edge-transitive graphs. It is also worth 
noting that the ||vj|| < 5 condition when applied to vectors {n e } eg£ ; arising from a graph 
simply means that the effective resistances of all edges are bounded; thus, we would be able 
to conclude that any graph with sufficiently small resistances can be split into two graphs 
that approximate it spectrally. 
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